Sfnet: Faster and Accurate Semantic Segmentation Via Semantic Flow

Abstract In this paper, we focus on exploring effective methods for faster and accurate semantic segmentation. A common practice to improve the performance is attain high-resolution feature maps with strong representation. Two strategies are widely used: atrous convolutions pyramid fusion, while both either computationally intensive or ineffective. Inspired by Optical Flow motion alignment between adjacent video frames, propose a Alignment Module (FAM) learn Semantic of levels broadcast high-level features effectively efficiently. Furthermore, integrating our FAM standard structure exhibits superior over other real-time methods, even lightweight backbone networks, such as ResNet-18 DFNet. Then further speed up inference procedure, also present novel Gated Dual directly align low-resolution where term improved version network SFNet-Lite. Extensive experiments conducted several challenging datasets, results show effectiveness SFNet particular, when using Cityscapes test set, SFNet-Lite series achieve 80.1 mIoU running at 60 FPS 78.8 120 STDC RTX-3090. Moreover, unify four driving datasets (i.e., Cityscapes, Mapillary, IDD, BDD) into one large dataset, which named Unified Driving Segmentation (UDS) dataset. It contains diverse domain style information. We benchmark representative works UDS. Both still best accuracy trade-off UDS, serves baseline in setting. The code models publicly available https://github.com/lxtGH/SFSegNets .